Overview

We manually curated 500\(+\) multiple modality PPMI MRI subjects. 5 messages from this effort.

Raw data quality is overall high

visual inspection by humans

Processing improves quality/usability further

failure rates humans vs processed
we provide scientists handles on this so that can be “strict” or “loose” in data selection
we document these parameters

Detailed curation results provide a unique resource in itself

for algorithm developers
- we ourselves benefit from this (will show examples)
for scientists
- to have confidence (or at least context regarding quality) in looking at automated IDPs derived from these data

Enables investigation of the “meaning” of M3RI at the individual level
Empowers investigation of the “meaning” of M3RI at the population level

the PPMI500 cohort

Reviewing multiple modality data quality for current phase of collection
we collaborated on cohort definition
- first phase - select subjects based on sampling from sites
- second phase - select subjects based on population characteristics (MJFF/PPMI)
show table of summary cohort

Table 1. Baseline cohort for subjects with T1w IDPs and non-missing diagnosis.
	CN	GenAR	LRRK2PD	SporadicAR	SporadicPD	p
	(N=47)	(N=1)	(N=4)	(N=230)	(N=217)	p
age	63.5 ± 12.5	71.9 ± NA	59.3 ± 7.1	67.7 ± 5.7	64.5 ± 9.4	< 0.001
Sex						0.026
Female	23 (48.9%)	1 (100.0%)	3 (75.0%)	102 (44.3%)	72 (33.2%)
Male	24 (51.1%)	0 ( 0.0%)	1 (25.0%)	128 (55.7%)	145 (66.8%)
duration.yrs			2.1 ± 0.7		1.0 ± 0.7	0.002
updrs.totscore	5.3 ± 3.8	14.0 ± NA	30.3 ± 13.1	12.1 ± 9.3	36.5 ± 15.0	< 0.001
CSFSAA						< 0.001
Negative	36 (85.7%)	1 (100.0%)	2 (50.0%)	82 (38.7%)	10 ( 4.9%)
Positive	6 (14.3%)	0 ( 0.0%)	2 (50.0%)	130 (61.3%)	194 (95.1%)
LEDD	0.0 ± 0.0	0.0 ± NA	259.0 ± 317.7	1.3 ± 19.8	3.6 ± 30.3	< 0.001
hasDTI						0.117
FALSE	9 (19.1%)	1 (100.0%)	1 (25.0%)	57 (24.8%)	38 (17.5%)
TRUE	38 (80.9%)	0 ( 0.0%)	3 (75.0%)	173 (75.2%)	179 (82.5%)
hasfMRI						0.003
FALSE	5 (10.6%)	1 (100.0%)	0 ( 0.0%)	12 ( 5.2%)	15 ( 6.9%)
TRUE	42 (89.4%)	0 ( 0.0%)	4 (100.0%)	218 (94.8%)	202 (93.1%)

1. Data quality is overall high

How do we define M3RI quality?

we manually QC’d each modality for each of 500 baseline subjects
- characterize reasons for failure

what is the QC protocol?

Review QC protocol – generally speaking

what are the QC results?

Summarize QC results for each modality
- raw data QC and failure rates
Summarize results for “joint” quality - how many subjects are high quality across all modalities at once?

Tables of Manual QC results

Table 2 Patterns of QC results by modality: FLAIR
	T2Flair	p
	(N=549)	p
qcfail_orientation
True	1 (100.0%)
qcfail_intensity_saturation
True	4 (100.0%)
qcfail_background_noise
True	6 (100.0%)
qcfail_motion
True	10 (100.0%)
qcfail_other
True	2 (100.0%)
qchuman_FLAIR		< 0.001
False	25 ( 4.6%)
True	524 (95.4%)
has_humanqc
1	549 (100.0%)

Table 3 Patterns of QC results by modality: T1w
	T1w	p
	(N=523)	p
qcfail_background_noise
True	2 (100.0%)
qcfail_motion
True	4 (100.0%)
qcfail_other
True	4 (100.0%)
qchuman_T1w		< 0.001
False	10 ( 1.9%)
True	513 (98.1%)
has_humanqc
1	523 (100.0%)

Table 4 Patterns of QC results by modality: NM
	Neuromelanin	p
	(N=502)	p
NM_QC_Ratings_Z		< 0.001
0	88 (18.3%)
1	393 (81.7%)
NM_QC_Ratings_SNVol		< 0.001
0	7 ( 1.5%)
1	474 (98.5%)
NM_QC_Ratings_AvgIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_SDIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_MaxIntensity		< 0.001
0	5 ( 1.0%)
1	473 (99.0%)
NM_QC_Ratings_Failures		< 0.001
0	386 (80.8%)
1	81 (16.9%)
2	6 ( 1.3%)
3	4 ( 0.8%)
4	1 ( 0.2%)

Table 5 Patterns of QC results by modality: DTI
	DTIb0	DTIdwi	p
	(N=1204)	(N=1340)	p
qcfail_orientation
True	16 (100.0%)	16 (100.0%)
qcfail_phantom
True	6 (100.0%)	6 (100.0%)
qcfail_intensity_saturation
True	0 ( 0.0%)	4 (100.0%)
qcfail_background_noise
True	50 (100.0%)	54 (100.0%)
qcfail_motion
True	54 (100.0%)	70 (100.0%)
qcfail_signal_dropout
True	14 (100.0%)	22 (100.0%)
qcfail_other			1.000
GHOSTING	10 (62.5%)	10 (62.5%)
True	6 (37.5%)	6 (37.5%)
qchuman_DTI			0.481
False	94 ( 7.8%)	116 ( 8.7%)
True	1110 (92.2%)	1224 (91.3%)
has_humanqc
1	1204 (100.0%)	1340 (100.0%)

Table 6 Patterns of QC results by modality: rsfMRI
	rsfMRI	rsfMRI_LR	rsfMRI_RL	p
	(N=2969)	(N=187)	(N=854)	p
qcfail_orientation
True	9 (100.0%)	1 (100.0%)	16 (100.0%)
qcfail_phantom
True	14 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_spacing
True	14 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_intensity_saturation
True	20 (100.0%)	10 (100.0%)	32 (100.0%)
qcfail_motion
True	581 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_signal_dropout
True	165 (100.0%)	15 (100.0%)	48 (100.0%)
qcfail_temporal_noise
True	1 (100.0%)	1 (100.0%)	8 (100.0%)
qcfail_insufficient_volumes
True	12 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_FOV
True	1 (100.0%)	0 ( 0.0%)	0 ( 0.0%)
qcfail_other				< 0.001
472 volumes, restart appended to initial scan?	0 ( 0.0%)	0 ( 0.0%)	8 (33.3%)
axial slices look wrong way around?	0 ( 0.0%)	1 (50.0%)	8 (33.3%)
strange artifacts in tsnr	0 ( 0.0%)	1 (50.0%)	8 (33.3%)
True	1 (50.0%)	0 ( 0.0%)	0 ( 0.0%)
wrap-around artifact	1 (50.0%)	0 ( 0.0%)	0 ( 0.0%)
qchuman_rsfMRI				0.002
False	457 (15.4%)	19 (10.2%)	96 (11.2%)
True	2512 (84.6%)	168 (89.8%)	758 (88.8%)
has_humanqc
1	2969 (100.0%)	187 (100.0%)	854 (100.0%)

T1w results

T1w QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
T1w	523	513	10	98.08795	1.9120459	qchuman_T1w
T1w	523	521	2	99.61759	0.3824092	qcfail_background_noise
T1w	523	519	4	99.23518	0.7648184	qcfail_motion
T1w	523	519	4	99.23518	0.7648184	qcfail_other

Flair results

FLAIR QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
T2Flair	549	524	25	95.44627	4.5537341	qchuman_FLAIR
T2Flair	549	548	1	99.81785	0.1821494	qcfail_orientation
T2Flair	549	545	4	99.27140	0.7285974	qcfail_intensity_saturation
T2Flair	549	543	6	98.90710	1.0928962	qcfail_background_noise
T2Flair	549	539	10	98.17851	1.8214936	qcfail_motion
T2Flair	549	547	2	99.63570	0.3642987	qcfail_other

DT results

DTI QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
DTIb0	1204	1110	94	92.19269	7.8073090	qchuman_DTI
DTIdwi	1340	1224	116	91.34328	8.6567164	qchuman_DTI
DTIb0	1204	1188	16	98.67110	1.3289037	qcfail_orientation
DTIdwi	1340	1324	16	98.80597	1.1940299	qcfail_orientation
DTIb0	1204	1198	6	99.50166	0.4983389	qcfail_phantom
DTIdwi	1340	1334	6	99.55224	0.4477612	qcfail_phantom
DTIb0	1204	1204	0	100.00000	0.0000000	qcfail_intensity_saturation
DTIdwi	1340	1336	4	99.70149	0.2985075	qcfail_intensity_saturation
DTIb0	1204	1154	50	95.84718	4.1528239	qcfail_background_noise
DTIdwi	1340	1286	54	95.97015	4.0298507	qcfail_background_noise
DTIb0	1204	1150	54	95.51495	4.4850498	qcfail_motion
DTIdwi	1340	1270	70	94.77612	5.2238806	qcfail_motion
DTIb0	1204	1190	14	98.83721	1.1627907	qcfail_signal_dropout
DTIdwi	1340	1318	22	98.35821	1.6417910	qcfail_signal_dropout
DTIb0	1204	1188	16	98.67110	1.3289037	qcfail_other
DTIdwi	1340	1324	16	98.80597	1.1940299	qcfail_other

rsfMRI results

rsfMRI QC summary
modality	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
rsfMRI	2969	2512	457	84.60761	15.3923880	qchuman_rsfMRI
rsfMRI_LR	187	168	19	89.83957	10.1604278	qchuman_rsfMRI
rsfMRI_RL	854	758	96	88.75878	11.2412178	qchuman_rsfMRI
rsfMRI	2969	2960	9	99.69687	0.3031324	qcfail_orientation
rsfMRI_LR	187	186	1	99.46524	0.5347594	qcfail_orientation
rsfMRI_RL	854	838	16	98.12646	1.8735363	qcfail_orientation
rsfMRI	2969	2955	14	99.52846	0.4715392	qcfail_phantom
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_phantom
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_phantom
rsfMRI	2969	2955	14	99.52846	0.4715392	qcfail_spacing
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_spacing
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_spacing
rsfMRI	2969	2949	20	99.32637	0.6736275	qcfail_intensity_saturation
rsfMRI_LR	187	177	10	94.65241	5.3475936	qcfail_intensity_saturation
rsfMRI_RL	854	822	32	96.25293	3.7470726	qcfail_intensity_saturation
rsfMRI	2969	2388	581	80.43112	19.5688784	qcfail_motion
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_motion
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_motion
rsfMRI	2969	2804	165	94.44257	5.5574267	qcfail_signal_dropout
rsfMRI_LR	187	172	15	91.97861	8.0213904	qcfail_signal_dropout
rsfMRI_RL	854	806	48	94.37939	5.6206089	qcfail_signal_dropout
rsfMRI	2969	2968	1	99.96632	0.0336814	qcfail_temporal_noise
rsfMRI_LR	187	186	1	99.46524	0.5347594	qcfail_temporal_noise
rsfMRI_RL	854	846	8	99.06323	0.9367681	qcfail_temporal_noise
rsfMRI	2969	2957	12	99.59582	0.4041765	qcfail_insufficient_volumes
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_insufficient_volumes
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_insufficient_volumes
rsfMRI	2969	2968	1	99.96632	0.0336814	qcfail_FOV
rsfMRI_LR	187	187	0	100.00000	0.0000000	qcfail_FOV
rsfMRI_RL	854	854	0	100.00000	0.0000000	qcfail_FOV

NM results

NM QC summary
NM.QC.cat	Total	Successes	Failures	Percent_Success	Percent_Failure	failcat
overall	478	386	92	80.75314	19.246862	NM_QC_Ratings_Failures
z.coord	478	391	87	81.79916	18.200837	NM_QC_Ratings_Z
sn.vol	478	471	7	98.53556	1.464435	NM_QC_Ratings_SNVol
avg.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_AvgIntensity
sd.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_SDIntensity
max.int	478	473	5	98.95397	1.046025	NM_QC_Ratings_MaxIntensity

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

we ourselves benefit from this (will show examples)

for scientists

to have confidence (or at least context regarding quality) in looking at automated IDPs derived from these data

3. Processing improves quality/usability further

effect of processing by modality

for each, summarize how processing improves the data
- rsfMRI is “easy” to see - throw away motion corrupted data
- developed methodology specific for PPMI that was informed by this process
- automated QC for these modalities for future work

we provide scientists handles on this so that can be “strict” or “loose” in data selection

we document these parameters

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

## [1] "     <0> BUILD-V <0> BUILD-V <0> BUILD-V <0> BUILD-V <0>    "
## [1] "initialDataTerm: -0.385290128127092  <o> mixer: pca  <o> E:  cca"
## [1] "Iteration: 1 bestEv: -0.0190496387336431 bestIt: 1 CE: -0.0190496387336431"
## [1] "Iteration: 2 bestEv: -0.152578040948536 bestIt: 2 CE: -0.152578040948536"
## [1] "Iteration: 3 bestEv: -0.152578040948536 bestIt: 2 CE: -0.129302661068566"
## [1] "Iteration: 4 bestEv: -0.15828931268243 bestIt: 4 CE: -0.15828931268243"
## [1] "Iteration: 5 bestEv: -0.186441610312862 bestIt: 5 CE: -0.186441610312862"
## [1] "Iteration: 6 bestEv: -0.186441610312862 bestIt: 5 CE: -0.156780368305935"
## [1] "Iteration: 7 bestEv: -0.186441610312862 bestIt: 5 CE: -0.15972787930179"
## [1] "Iteration: 8 bestEv: -0.186441610312862 bestIt: 5 CE: -0.159945570074572"
## [1] "Iteration: 9 bestEv: -0.186441610312862 bestIt: 5 CE: -0.159945570074572"
## [1] "Iteration: 10 bestEv: -0.186441610312862 bestIt: 5 CE: -0.159945570074572"
## [1] "Iteration: 11 bestEv: -0.186441610312862 bestIt: 5 CE: -0.179384499667904"
## [1] 0.04241557
## [1] 0.0658389
## [1] 0.04042775
## [1] "1 1.24046922113173e-05"

## [1] "perm 35342"
## [1]  62   3 335 334  46 308
## [1] 316 206 238 162 241 328
## [1] 270 200 179 124 297 292
## [1]  45 138  74 165 201 276
## [1] 0.04889521
## [1] 0.05616091
## [1] 0.04830412
## [1] "1 0.0531580647417864"

## [1] "perm 26048"
## [1]  93 260  83 336 296 320
## [1]   3 101 281 337  68 133
## [1] 116 100  45   3  46 313
## [1]  88  28 119  43 187 312
## [1] 0.0361035
## [1] 0.06172612
## [1] 0.09639387
## [1] "1 0.360563451242297"

## [1] "perm 97416"
## [1] 295 172 132 325 170  63
## [1] 343 216 101 318  74  58
## [1]  17 218 124 267 304 183
## [1]  75 255 276 175 152 119
## [1] 0.0525526
## [1] 0.06426883
## [1] 0.0204605
## [1] "1 0.0574346126483664"

## [1] "perm 24447"
## [1] 328 134 265  92  91 275
## [1]  92 167 261 195  69   1
## [1] 163 221 125 167  27 175
## [1]  96 305 114 288 160 124
## [1] 0.1010434
## [1] 0.03955074
## [1] 0.03840789
## [1] "1 0.428960232006515"

## [1] "perm 77422"
## [1] 127 249 196 168 259  24
## [1]   9 220 233   4 122 133
## [1] 132 128 258 174 344 124
## [1] 322  97 345 268  31 140
## [1] 0.05513479
## [1] 0.03632016
## [1] 0.0135581
## [1] "1 0.0340373188659457"

## [1] "perm 1114"
## [1] 121 108  36 232 320  73
## [1]  86 128 237  25 107 244
## [1]  15 165  96  88 157 184
## [1] 234 291 160 297 301 129
## [1] 0.02194891
## [1] 0.03828378
## [1] 0.04038219
## [1] "1 0.000578757078941461"

## [1] "perm 6882"
## [1]   1  38 147 114 329 183
## [1] 243 233 251 154 127 315
## [1] 148 199  38  93 162 313
## [1] 109 101 324 269 145 329
## [1] 0.08109294
## [1] 0.06851608
## [1] 0.05499789
## [1] "1 0.00135110655593116"

## [1] "perm 32574"
## [1] 322  69 116 252 170 304
## [1]  33  27 253 291 199  79
## [1] 122  48  25 260 202  81
## [1]  52 175 118 153 271  25
## [1] 0.04780283
## [1] 0.06309846
## [1] 0.08886985
## [1] "1 0.363325596759259"

## [1] "perm 57805"
## [1] 296 233 129 229 145   6
## [1] 199 333  57 116 140 148
## [1]  70 215 336 198  85 232
## [1]  80 248  33 284 214  48
## [1] 0.08105317
## [1] 0.09904561
## [1] 0.04559614
## [1] "1 0.0100515863486305"

## [1] "perm 26491"
## [1] 299 210 236 270 325 276
## [1] 344 254 101 270 138 310
## [1] 157 320 306  93 336  29
## [1] 195 334  70   4 287  36
## [1] 0.004745701
## [1] 0.07711808
## [1] 0.05285046
## [1] "1 0.0435032471658854"

PPMI500 data curation effort

Brian B. Avants^1,2

Lino Becerra¹

Leon Fonville^2,†

Olivia Hampton^1,†

Alexandra Reardon^1,†

Andrew Stenger^1,†

Xue Wang^1,†

Roger Gunn

Overview

the PPMI500 cohort

1. Data quality is overall high

How do we define M3RI quality?

what is the QC protocol?

what are the QC results?

Tables of Manual QC results

T1w results

Flair results

DT results

rsfMRI results

NM results

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

for scientists

3. Processing improves quality/usability further

effect of processing by modality

we provide scientists handles on this so that can be “strict” or “loose” in data selection

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

PPMI500 data curation effort

Brian B. Avants1,2

Lino Becerra1

Leon Fonville2,†

Olivia Hampton1,†

Alexandra Reardon1,†

Andrew Stenger1,†

Xue Wang1,†

Roger Gunn

Overview

the PPMI500 cohort

1. Data quality is overall high

How do we define M3RI quality?

what is the QC protocol?

what are the QC results?

Tables of Manual QC results

T1w results

Flair results

DT results

rsfMRI results

NM results

do a check based on rsfmri motion – if the qc data “makes sense”

do a check based on DTI – if the qc data “makes sense”

do a check based on NM – if the qc data “makes sense”

do a check based on FLAIR – if the qc data “makes sense”

do a check based on T1w – if the qc data “makes sense”

2. Detailed curation results provide a unique resource in itself

for algorithm developers

for scientists

3. Processing improves quality/usability further

effect of processing by modality

we provide scientists handles on this so that can be “strict” or “loose” in data selection

4. Enables investigation of the “meaning” of M3RI at the individual level

5. Empowers investigation of the “meaning” of M3RI at the population level

simlr

Brian B. Avants^1,2

Lino Becerra¹

Leon Fonville^2,†

Olivia Hampton^1,†

Alexandra Reardon^1,†

Andrew Stenger^1,†

Xue Wang^1,†